Used to Be a Dime, Now It's a Dollar: Revised Speech Perception in Noise Key Word Predictability Revisited 40 Years On.

PURPOSE: Almost 40 years after its development, in this article, we reexamine the relevance and validity of the ubiquitously used Revised Speech Perception in Noise (R-SPiN) sentence corpus. The R-SPiN corpus includes "high-context" and "low-context" sentences and has been widely used in the field of hearing research to examine the benefit derived from semantic context across English-speaking listeners, but research investigating age differences has yielded somewhat inconsistent findings. We assess the appropriateness of the corpus for use today in different English-language cultures (i.e., British and American) as well as for older and younger adults. METHOD: Two hundred forty participants, including older (60-80 years) and younger (19-31 years) adult groups in the the United Kingdom and United States, completed a cloze task consisting of R-SPiN sentences with the final word removed. Cloze, as a measure of predictability, and entropy, as a measure of response uncertainty, were compared between culture and age groups. RESULTS: Most critically, of the 200 "high-context" stimuli, only around half were assessed as highly predictable for older adults (United Kingdom: 109; United States: 107); and fewer still, for younger adults (United Kingdom: 75; United States: 81). We also found dominant responses to these "high-context" stimuli varied between cultures, with U.S. responses being more likely to match the original R-SPiN target. CONCLUSIONS: Our findings highlight the issue of incomplete transferability of corpus items across English-language cultures as well as diminished equivalency for older and younger adults. By identifying relevant items for each population, this work could facilitate the interpretation of inconsistent findings in the literature, particularly relating to age effects.

The effect of metacognitive executive function training on children's executive function, proactive control, and academic skills.

The current study investigated the effects of metacognitive and executive function (EF) training on childhood EF (inhibition, working memory [WM], cognitive flexibility, and proactive/reactive control) and academic skills (reading, reasoning, and math) among children from disadvantaged backgrounds. Children (N = 134, Mage = 8.70 years) were assigned randomly to the three training groups: (a) metacognitive training of basic EF processes (meta-EF), (b) training of basic EF processes (basic-EF), and (c) active controls (active control). They underwent 16 training sessions over the course of 2 months. No effects of EF and/or metacognitive training were found for academic outcomes. However, both meta-EF and basic-EF groups demonstrated greater gains than the active control group on proactive control engagement and WM, suggesting that EF training promotes a shift to more mature ways of engaging EF. Our findings suggest minimal near- and far-transfer effects of metacognitive training but highlight that proactive engagement of EF can be promoted through EF training in children. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations.

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

Alpha Oscillations During Effortful Continuous Speech: From Scalp EEG to Ear-EEG.

OBJECTIVE: The purpose of this study was to investigate alpha power as an objective measure of effortful listening in continuous speech with scalp and ear-EEG. METHODS: Scalp and ear-EEG were recorded simultaneously during presentation of a 33-s news clip in the presence of 16-talker babble noise. Four different signal-to-noise ratios (SNRs) were used to manipulate task demand. The effects of changes in SNR were investigated on alpha event-related synchronization (ERS) and desynchronization (ERD). Alpha activity was extracted from scalp EEG using different referencing methods (common average and symmetrical bi-polar) in different regions of the brain (parietal and temporal) and ear-EEG. RESULTS: Alpha ERS decreased with decreasing SNR (i.e., increasing task demand) in both scalp and ear-EEG. Alpha ERS was also positively correlated to behavioural performance which was based on the questions regarding the contents of the speech. CONCLUSION: Alpha ERS/ERD is better suited to track performance of a continuous speech than listening effort. SIGNIFICANCE: EEG alpha power in continuous speech may indicate of how well the speech was perceived and it can be measured with both scalp and Ear-EEG.

Conversation success in one-to-one and group conversation: a group concept mapping study of adults with normal and impaired hearing.

OBJECTIVE: The concept of conversation success is undefined, although prior work has variously related it to accurate exchange of information, alignment between interlocutors, and good management of misunderstandings. This study aimed (1) to identify factors of conversation success and (2) to explore the importance of these factors in one-to-one versus group conversations. DESIGN: Group concept mapping method was applied. Participants responded to two brainstorming prompts ("What does 'successful conversation' look like?" and "Think about a successful conversation you have taken part in. What aspects of that conversation contributed to its success?"). The resulting statements were sorted into related clusters and rated in importance for one-to-one and group conversation. STUDY SAMPLE: Thirty-five adults with normal and impaired hearing. RESULTS: Seven clusters were identified: (1) Being able to listen easily; (2) Being spoken to in a helpful way; (3) Being engaged and accepted; (4) Sharing information as desired; (5) Perceiving flowing and balanced interaction; (6) Feeling positive emotions; (7) Not having to engage coping mechanisms. Three clusters (1, 2, and 4) were more important in group than in one-to-one conversation. There were no differences by hearing group. CONCLUSIONS: These findings emphasise that conversation success is a multifaceted concept.

Timing of head turns to upcoming talkers in triadic conversation: Evidence for prediction of turn ends and interruptions.

In conversation, people are able to listen to an utterance and respond within only a few hundred milliseconds. It takes substantially longer to prepare even a simple utterance, suggesting that interlocutors may make use of predictions about when the talker is about to end. But it is not only the upcoming talker that needs to anticipate the prior talker ending-listeners that are simply following the conversation could also benefit from predicting the turn end in order to shift attention appropriately with the turn switch. In this paper, we examined whether people predict upcoming turn ends when watching conversational turns switch between others by analysing natural conversations. These conversations were between triads of older adults in different levels and types of noise. The analysis focused on the observer during turn switches between the other two parties using head orientation (i.e. saccades from one talker to the next) to identify when their focus moved from one talker to the next. For non-overlapping utterances, observers started to turn to the upcoming talker before the prior talker had finished speaking in 17% of turn switches (going up to 26% when accounting for motor-planning time). For overlapping utterances, observers started to turn towards the interrupter before they interrupted in 18% of turn switches (going up to 33% when accounting for motor-planning time). The timing of head turns was more precise at lower than higher noise levels, and was not affected by noise type. These findings demonstrate that listeners in natural group conversation situations often exhibit head movements that anticipate the end of one conversational turn and the beginning of another. Furthermore, this work demonstrates the value of analysing head movement as a cue to social attention, which could be relevant for advancing communication technology such as hearing devices.

Midfrontal theta oscillations and conflict monitoring in children and adults.

Conflict monitoring is central in cognitive control, as detection of conflict serves as a signal for the need to engage control. This study examined whether (1) midfrontal theta oscillations similarly support conflict monitoring in children and adults, and (2) performance monitoring difficulty influences conflict monitoring and resolution. Children (n = 25) and adults (n = 24) completed a flanker task with fair or rigged response feedback. Relative to adults, children showed a smaller congruency effect on midfrontal theta power, overall lower midfrontal theta power and coherence, and (unlike adults) no correlation between midfrontal theta power and N2 amplitude, suggesting that reduced neural communication efficiency contributes to less efficient conflict monitoring in children than adults. In both age groups, response feedback fairness affected response times and the P3, but neither midfrontal theta oscillations nor the N2, indicating that performance monitoring difficulty influenced conflict resolution but not conflict monitoring.

Synchrony as a measure of conversation difficulty: Movement coherence increases with background noise level and complexity in dyads and triads.

When people interact, they fall into synchrony. This synchrony has been demonstrated in a range of contexts, from walking or playing music together to holding a conversation, and has been linked to prosocial outcomes such as development of rapport and efficiency of cooperation. While the basis of synchrony remains unclear, several studies have found synchrony to increase when an interaction is made challenging, potentially providing a means of facilitating interaction. Here we focus on head movement during free conversation. As verbal information is obscured when conversing over background noise, we investigate whether synchrony is greater in high vs low levels of noise, as well as addressing the effect of background noise complexity. Participants held a series of conversations with unfamiliar interlocutors while seated in a lab, and the background noise level changed every 15-30s between 54, 60, 66, 72, and 78 dB. We report measures of head movement synchrony recorded via high-resolution motion tracking at the extreme noise levels (i.e., 54 vs 78 dB) in dyads (n = 15) and triads (n = 11). In both the dyads and the triads, we report increased movement coherence in high compared to low level speech-shaped noise. Furthermore, in triads we compare behaviour in speech-shaped noise vs multi-talker babble, and find greater movement coherence in the more complex babble condition. Key synchrony differences fall in the 0.2-0.5 Hz frequency bands, and are discussed in terms of their correspondence to talkers' average utterance durations. Additional synchrony differences occur at higher frequencies in the triads only (i.e., >5 Hz), which may relate to synchrony of backchannel cues (as multiple individuals were listening and responding to the same talker). Not only do these studies replicate prior work indicating interlocutors' increased reliance on behavioural synchrony as task difficulty increases, but they demonstrate these effects using multiple difficulty manipulations and across different sized interaction groups.

The effects of dual-task interference in predicting turn-ends in speech and music.

Determining when a partner's spoken or musical turn will end requires well-honed predictive abilities. Evidence suggests that our motor systems are activated during perception of both speech and music, and it has been argued that motor simulation is used to predict turn-ends across domains. Here we used a dual-task interference paradigm to investigate whether motor simulation of our partner's action underlies our ability to make accurate turn-end predictions in speech and in music. Furthermore, we explored how specific this simulation is to the action being predicted. We conducted two experiments, one investigating speech turn-ends, and one investigating music turn-ends. In each, 34 proficient pianists predicted turn-endings while (1) passively listening, (2) producing an effector-specific motor activity (mouth/hand movement), or (3) producing a task- and effector-specific motor activity (mouthing words/fingering a piano melody). In the speech experiment, any movement during speech perception disrupted predictions of spoken turn-ends, whether the movement was task-specific or not. In the music experiment, only task-specific movement (i.e., fingering a piano melody) disrupted predictions of musical turn-ends. These findings support the use of motor simulation to make turn-end predictions in both speech and music but suggest that the specificity of this simulation may differ between domains.

Conversation in small groups: Speaking and listening strategies depend on the complexities of the environment and group.

Many conversations in our day-to-day lives are held in noisy environments - impeding comprehension, and in groups - taxing auditory attention-switching processes. These situations are particularly challenging for older adults in cognitive and sensory decline. In noisy environments, a variety of extra-linguistic strategies are available to speakers and listeners to facilitate communication, but while models of language account for the impact of context on word choice, there has been little consideration of the impact of context on extra-linguistic behaviour. To address this issue, we investigate how the complexity of the acoustic environment and interaction situation impacts extra-linguistic conversation behaviour of older adults during face-to-face conversations. Specifically, we test whether the use of intelligibility-optimising strategies increases with complexity of the background noise (from quiet to loud, and in speech-shaped vs. babble noise), and with complexity of the conversing group (dyad vs. triad). While some communication strategies are enhanced in more complex background noise, with listeners orienting to talkers more optimally and moving closer to their partner in babble than speech-shaped noise, this is not the case with all strategies, as we find greater vocal level increases in the less complex speech-shaped noise condition. Other behaviours are enhanced in the more complex interaction situation, with listeners using more optimal head orientations, and taking longer turns when gaining the floor in triads compared to dyads. This study elucidates how different features of the conversation context impact individuals' communication strategies, which is necessary to both develop a comprehensive cognitive model of multimodal conversation behaviour, and effectively support individuals that struggle conversing.

Listeners are better at predicting speakers similar to themselves.

Although it takes several hundred milliseconds to prepare a spoken contribution, gaps between turns in conversation tend to be much shorter. To produce these short gaps, it appears that interlocutors predict the end of their partner's turn. The theory of prediction-by-simulation proposes that individuals use their own motor system to model a partner's upcoming actions by referring to prior production experience. In this study we investigate the role of motor experience for both predicting a turn-end and producing a spoken response by manipulating the similarity of heard speech to participants' own production style. We hypothesised that they would be better at predicting, and initiating responses to, speech produced in the style they speak themselves. Participants recorded a series of questions in two sessions, and several months later they listened to their own speech and that of a stylistically similar and a stylistically dissimilar participant (as assessed by independent raters). Participants predicted the end of 60 of these questions by pressing a button, and for the remaining 60 questions, by producing a spoken response. An analysis of response times showed that participants' button-press responses were faster for utterances spoken by themselves and by a stylistically similar partner, than for utterances spoken by a stylistically dissimilar partner. We conclude that simulation facilitates prediction of similar speakers.

Consistent use of proactive control and relation with academic achievement in childhood.

As children become older, they better maintain task-relevant information in preparation of upcoming cognitive demands. This is referred to as proactive control, which is a key component of cognitive control development. However, it is still uncertain whether children engage in proactive control consistently across different contexts and how proactive control relates to academic abilities. This study used two common tasks-the AX Continuous Performance Task (AX-CPT) and the Cued Task-Switching Paradigm (CTS)-to examine whether proactive control engagement in 102 children (age range: 6.91-10.91 years) converges between the two tasks and predicts academic abilities. Proactive control indices modestly correlated between tasks in higher but not lower working-memory children, suggesting that consistency in proactive control engagement across contexts is relatively low during childhood but increases with working memory capacity. Further, working memory (but not verbal speed) predicted proactive control engagement in both tasks. While proactive control as measured by each task predicted math and reading performance, only proactive control measured by CTS additionally predicted reasoning, suggesting that proactive control can be used as a proxy for academic achievements.

Encouraging performance monitoring promotes proactive control in children.

Monitoring progression towards one's goals is essential for efficient cognitive control. Immature performance monitoring may contribute to suboptimal cognitive control engagement in childhood, potentially explaining why children engage control reactively even when proactive control would be more effective. This study investigated whether encouraging children to actively monitor their performance results in more mature control engagement. Electroencephalography data were collected while children and adults performed a flanker task in three conditions in which they were provided no feedback, standard feedback, or were asked to estimate their own feedback. Both age groups accurately estimated their own feedback. Critically, feedback estimation promoted online performance monitoring and proactive engagement of attention and inhibition during the flanker period in children. These findings indicate that proactive control engagement in childhood can be effectively supported by encouraging performance monitoring.

Age-related differentiation in verbal and visuospatial working memory processing in childhood.

Working memory (WM), a key feature of the cognitive system, allows for maintaining and processing information simultaneously and in a controlled manner. WM processing continuously develops across childhood, with significant increases both in verbal and visuospatial WM. Verbal and visuospatial WM may show different developmental trajectories, as verbal (but not visuospatial) WM relies on internal verbal rehearsal, which is less developed in younger children. We examined complex VWM and VSWM performance in 125 younger (age 4-6 years) and 101 older (age 8-10 years) children. Latent multi-group modeling showed that (1) older children performed better on both verbal and visuospatial WM span tasks than younger children, (2) both age groups performed better on verbal than visuospatial WM, and (3) a model with two factors representing verbal and visuospatial WM fit the data better than a one-factor model. Importantly, the correlation between the two factors was significantly higher in younger than in older children, suggesting an age-related differentiation of verbal and spatial WM processing in middle childhood. Age-related differentiation is an important characteristic of cognitive functioning and thus the findings contribute to our general understanding of WM processing.

Speech, movement, and gaze behaviours during dyadic conversation in noise.

How do people have conversations in noise and make themselves understood? While many previous studies have investigated speaking and listening in isolation, this study focuses on the behaviour of pairs of individuals in an ecologically valid context. Specifically, we report the fine-grained dynamics of natural conversation between interlocutors of varying hearing ability (n = 30), addressing how different levels of background noise affect speech, movement, and gaze behaviours. We found that as noise increased, people spoke louder and moved closer together, although these behaviours provided relatively small acoustic benefit (0.32 dB speech level increase per 1 dB noise increase). We also found that increased noise led to shorter utterances and increased gaze to the speaker's mouth. Surprisingly, interlocutors did not make use of potentially beneficial head orientations. While participants were able to sustain conversation in noise of up to 72 dB, changes in conversation structure suggested increased difficulty at 78 dB, with a significant decrease in turn-taking success. Understanding these natural conversation behaviours could inform broader models of interpersonal communication, and be applied to the development of new communication technologies. Furthermore, comparing these findings with those from isolation paradigms demonstrates the importance of investigating social processes in ecologically valid multi-person situations.

The effects of verbal and spatial memory load on children's processing speed.

Examining the impact of maintenance on processing speed allows us to test whether storage and processing resources are shared. Comparing these relationships in children of different ages allows further insight into whether one or multiple resources for these operations must be assumed and whether remembering is proactive throughout childhood. We tested 185 4- to 6- and 8- to 10-year-old children using adaptive complex span tasks, in which simple judgments were interleaved between to-be-remembered items. The adaptiveness of our tasks ensured that all participants frequently correctly recalled the items. If storage and processing require a single resource, and if participants serially reactivate the memoranda between processing episodes, processing response times should increase with serial position of the processing judgment within lists. We observed different within-list dynamics for each age group. Older children's processing judgments slowed gradually when more than two memory items were maintained. By contrast, younger children showed no evidence of slower processing with increasing memory load. Our results support models of working memory that assume that some common resource is responsible for verbal and spatial storage and processing. They also support the notion that remembering becomes more proactive as children mature.

Determining the end of a musical turn: Effects of tonal cues.

Successful duetting requires that musicians coordinate their performance with their partners. In the case of turn-taking in improvised performance they need to be able to predict their partner's turn-end in order to accurately time their own entries. Here we investigate the cues used for accurate turn-end prediction in musical improvisations, focusing on the role of tonal structure. In a response-time task, participants more accurately determined the endings of (tonal) jazz than (non-tonal) free improvisation turns. Moreover, for the jazz improvisations, removing low frequency information (<2100Hz) - and hence obscuring the pitch relationships conveying tonality - reduced response accuracy, but removing high frequency information (>2100Hz) had no effect. Neither form of filtering affected response accuracy in the free improvisation condition. We therefore argue that tonal cues aided prediction accuracy for the jazz improvisations compared to the free improvisations. We compare our results with those from related speech research (De Ruiter et al., 2006), to draw comparisons between the structural function of tonality and linguistic syntax.

Incremental comprehension of pitch relationships in written music: Evidence from eye movements.

To investigate how proficient pianists comprehend pitch relationships in written music when they first encounter it we conducted two experiments in which proficient pianists' eyes were tracked while they read and played single-line melodies. In Experiment 1, participants played at their own speed; in Experiment 2 they played with an external metronome. The melodies were either congruent or anomalous, with the anomaly involving one bar being shifted in pitch to alter the implied harmonic structure (e.g., non-resolution of a dominant). In both experiments, anomaly led to rapid disruption in participants' eye-movements in terms of regressions from the target bar, indicating that pianists process written pitch relationships online. This is particularly striking because in musical sight-reading eye movement behaviour is constrained by the concurrent performance. Both experiments also showed that anomaly induced pupil dilation. Together these results indicate that proficient pianists rapidly integrate the music that they read into the prior context, and that anomalies in terms of pitch relationships lead to processing difficulty. These findings parallel those of text reading, suggesting that structural processing involves similar constraints across domains.

Causal Role of Motor Simulation in Turn-Taking Behavior.

Overlap between sensory and motor representations has been documented for a range of human actions, from grasping (Rizzolatti et al., 1996b) to playing a musical instrument (Novembre and Keller, 2014). Such overlap suggests that individuals use motor simulation to predict the outcome of observed actions (Wolpert, 1997). Here we investigate motor simulation as a basis of human communication. Using a musical turn-taking task, we show that pianists call on motor representations of their partner's part to predict when to come in for their own turn. Pianists played alternating solos with a videoed partner, and double-pulse transcranial magnetic stimulation was applied around the turn-switch to temporarily disrupt processing in two cortical regions implicated previously in different forms of motor simulation: (1) the dorsal premotor cortex (dPMC), associated with automatic motor resonance during passive observation of hand actions, especially when the actions are familiar (Lahav et al., 2007); and (2) the supplementary motor area (SMA), involved in active motor imagery, especially when the actions are familiar (Baumann et al., 2007). Stimulation of the right dPMC decreased the temporal accuracy of pianists' (right-hand) entries relative to sham when the partner's (left-hand) part had been rehearsed previously. This effect did not occur for dPMC stimulation without rehearsal or for SMA stimulation. These findings support the role of the dPMC in predicting the time course of observed actions via resonance-based motor simulation during turn-taking. Because turn-taking spans multiple modes of human interaction, we suggest that simulation is a foundational mechanism underlying the temporal dynamics of joint action. SIGNIFICANCE STATEMENT: Even during passive observation, seeing or hearing somebody execute an action from within our repertoire activates motor cortices of our brain. But what is the functional relevance of such "motor simulation"? By combining a musical duet task with a real-time repetitive transcranial magnetic stimulation protocol, we provide evidence indicating that the dorsal premotor cortex plays a causal role in accurate turn-taking coordination between a pianist and their observed interaction partner. Given that turn-taking behavior is a fundamental feature of human communication, we suggest that simulation is a foundational mechanism underlying the temporal dynamics of communicative joint action.

Perception of 'Back-Channeling' Nonverbal Feedback in Musical Duo Improvisation.

In witnessing face-to-face conversation, observers perceive authentic communication according to the social contingency of nonverbal feedback cues ('back-channeling') by non-speaking interactors. The current study investigated the generality of this function by focusing on nonverbal communication in musical improvisation. A perceptual experiment was conducted to test whether observers can reliably identify genuine versus fake (mismatched) duos from musicians' nonverbal cues, and how this judgement is affected by observers' musical background and rhythm perception skill. Twenty-four musicians were recruited to perform duo improvisations, which included solo episodes, in two styles: standard jazz (where rhythm is based on a regular pulse) or free improvisation (where rhythm is non-pulsed). The improvisations were recorded using a motion capture system to generate 16 ten-second point-light displays (with audio) of the soloist and the silent non-soloing musician ('back-channeler'). Sixteen further displays were created by splicing soloists with back-channelers from different duos. Participants (N = 60) with various musical backgrounds were asked to rate the point-light displays as either real or fake. Results indicated that participants were sensitive to the real/fake distinction in the free improvisation condition independently of musical experience. Individual differences in rhythm perception skill did not account for performance in the free condition, but were positively correlated with accuracy in the standard jazz condition. These findings suggest that the perception of back-channeling in free improvisation is not dependent on music-specific skills but is a general ability. The findings invite further study of the links between interpersonal dynamics in conversation and musical interaction.